Measuring Sociocultural Factors of Success in Information Quality Projects

Information and data quality practitioners are in general agreement that social, cultural, and organizational factors are the most important in determining the success or failure of an organization’s data quality programs. This paper presents some of the first research undertaken to substantiate these anecdotal claims. The paper describes a survey of recent graduates from the University of Arkansas at Little Rock Information Quality Graduate Program. In the survey the graduates rate how much influence these sociocultural factors had on the outcomes of their data quality projects. The results of the survey support the practitioners’ claims. Introduction Do social and cultural issues pose formidable challenges to information and data quality programs? Information is arguably the most valuable currency for organizational trade in today’s business environment. Fitness for use is a foundational tenet of information and data quality programs, and as we assess the fitness of our informational assets we recognize there are key factors which will either propel our information and data quality programs into a competitive force or stifle our competitive edge. Practitioners recognize that social, cultural, and organizational alignment are essential to the success of data quality programs; conversely that social, cultural, and organizational barriers present the most formidable challenges to such programs. Anecdotal evidence shows social and cultural factors within organizations are key drivers of success, or failure, of information and data quality programs. Relevant experience and research in manufacturing quality supports the proposition. But is this really true? Talburt, et.al proposed that this question is a current information quality research challenge. (1) This paper presents the first hard evidence. It summarizes results of a survey conducted of recent UALR master’s degree Information Quality graduates. While the survey has limitations, it generally supports the practitioners’ claims. In total, forty-six (of ninety-seven possible) graduates responded to a thirty-two question survey on their views of an improvement project they completed as part of their degree programs. Respondents almost uniformly cited factors such as “top management commitment,” and “communication with customers,” as critical to success, while “personnel competency” was cited less often. We have set out to develop a dynamic modeling and simulation of an organization’s information and data quality objectives and performance. The model and simulation should lead to recommendations for maximizing information assets by minimizing the impact of sociocultural factors. This study represents our first attempts to identify the sociocultural factors that should be incorporated into this model. It is based on prior work in the area of change management. 1 This paper is not intended to imply MITRE's concurrence with, or support for, the positions, opinions or viewpoints expressed by the authors. Literature Review The early work of Clotaire Rapaille on cultural archetypes became known to AT&T in the 1980s. As a quality manager in Network Systems, Marilyn Zuckerman began working with Rapaille on developing an American archetype of quality or the understanding of the subliminal thoughts that follow quality in the American culture. (2) Christine Robinson also wrote on these studies, along with other similar archetypal models, on how these ideas should be used by an organization that wants to improve the quality of their products. (3) While these ideas were originally researched for and implemented in the original quality arena – manufacturing; it stands to reason that an archetype of quality in any arena would carry over to data and information quality. In the 1990s, Peter Senge and his associates introduced an approach called organizational learning which started with the idea that organizations are the products of the ways that the people in them think and interact. (4) Instead of providing increased training in systems that will change, Senge advocates providing training for discovering these ways of thinking and interacting and thus how this will affect the behavior and performance of the system. He also uses different levels of explanations for describing the how and why of any event with the third level or structural explanations providing the most impact. This level of explanation addresses the underlying causes of behavior at a level that can be changed and because the structure in human systems includes the “operating policies” of the decision makers in the system, redesigning the decision making redesigns the system structure. This structure or archetype most applicable to organizational change is called “Limits to Growth”. In the Dance of Change model, there are a number of growth processes for profound change as well as challenges to profound change. The growth processes of profound change include: investment in change initiatives, development of learning capabilities, enhancing personal results, developing networks of committed people, and improving business results. The challenges to profound change fall into 3 categories: the challenges of initiating, the challenges of sustaining transformation, and the challenges of redesigning and rethinking. The challenges of initiating include: not enough time, no help (coaching and support), not relevant, and walking the talk. The challenges of sustaining include: fear and anxiety, assessment and measurement, and true believers and nonbelievers. The challenges of redesign and rethinking include: governance, diffusion, and strategy and purpose. The Limits to Growth archetype helps us see how the balance between these elements shifts over time. It particularly helps us come to terms with the ways in which, by pushing hard to overcome the constraints on our lives, we make the effects of those constraints even worse than they otherwise would be. We can become aware of the symptoms that are expressions of the way that structure is driving behavior. We can understand the mental models that are behind the way that the organization typically reacts, and we can craft appropriate strategies that give us higher leverage control actions to address the situations that confront us. Earlier work presented a review of several management and organizational models considered to be useful for directing change (5); however, the models that continue to drive this research remain those of Senge. We are interested in developing a dynamic systems model using the Senge Limits to Growth models and archetype. The initial ideas that have led to the current system dynamics approach for policy analysis and design were first published by Jay Forrester in 1958. (6) This approach emphasizes the multi-loop, multi-state, nonlinear character of the real world and we believe that it will allow us to uniquely define the dynamic complexities that are involved in any system of change. In order to proceed with mathematical modeling of the change models, we turned to other research for assistance in defining the variables that are considered critical to our problem of integrating information quality into the organization. In a 2013 study, Hongjiang Xu presented 25 critical success factors (CSFs) for data quality. (7) From these CSFs, we focused on the 14 factors that we determined to be based on organizational culture. These factors were then used as a basis for a survey and were listed as: 1) Top management commitment to Information Quality 2) Middle management commitment to Information Quality 3) Education and training 4) Clear IQ vision for the entire organization 5) Establish an IQ manager position to manage overall IQ 6) Appropriate organizational structure 7) IQ policies and standards 8) Organizational culture of focusing on IQ 9) Focus on information users' needs and their quality requirements 10) Effective employee relations 11) Management of changes 12) Continuous improvement 13) Teamwork/Communication 14) Personnel competency Data Collection and Methods In June and July of 2014, an invitation to participate in a survey was emailed to 97 graduates of the Masters in Information Quality Program at the University of Arkansas at Little Rock. The intent of this survey was to obtain information concerning the execution of the required Information Quality project each student managed and submitted as part of their graduation requirements. The survey was made available online via a link distributed in the invitation email and was open for 45 days. There were a total of 46 responses for a response rate of 47%. With this sample size, there is a confidence level of 90% with an 8.66% margin of error. (8) The survey consisted of 26 topic questions along with five (5) demographic questions and the opportunity to submit other open comments, totaling to 32 questions. Twenty-one of the 26 questions were based on a Likert scale in order to assess qualitative attitudes on the importance of cultural issues. The remaining five questions were multiple-choice questions. For purposes of this study, the following definitions were provided to the survey participants: Information Quality for this survey, means accurate, timely, complete, and consistent data. Users all users of the information in the area of your project. Could be internal and/or external to the organization. Top Management executive or senior management, includes the highest management positions in the organization. Middle Management is responsible for implementing the strategic decisions of top management. Middle managers make tactical/short-range decisions. Non-management employees includes production, clerical and staff personnel. The questions included in the survey are provided in figure 1 and as screenshots in Appendix A. 1. Are you a graduate of the UALR Information Quality Master's degree program? 2. Are you currently employed in the Information Quality field? 3. From an Information Quality standpoint, I was able to accomplish everything I desired with my project. 4. I believe that there were social/cultural issues NEGATIVELY impacting the success of my project. 5. I believe that there were social/cultural issues POSITIVELY impacting the success of my project. 6. I experienced top management commitment during my IQ project. 7. I experienced middle management commitment during my IQ project. 8. Education and training on the systems in the area of my project were provided to new personnel or when the systems were changed. 9. Education and training on the systems, in the area of my project, were provided regularly to existing employees and managers. 10. There was a data quality vision for the entire organization before/during my IQ project. (Sufficient funds are allocated, technical tools, expertise and skilled personnel ensure IQ) 11. There was an established DQ/IQ manager position within the organization to manage overall IQ. 12. There were appropriate (simple, relevant and consistent) IQ policies and standards. 13. There was a focus on information users' needs and their quality requirements before/during my IQ project. 14. There was high employee self-satisfaction, job security, and career development before/during my IQ project. 15. The organization handled the changes, suggested during my IQ project, well. 16. At this organization, employees work as a team and have good communication. 17. At this organization, there is good communication within a department. 18. At this organization, there is good communication between different departments. 19. At this organization, there is good communication between different professionals, such as between accounting and IT. 20. The people involved in my IQ project had a good understanding of the systems involved and of data quality. 21. In the organization, the organizational structure was clearly defined prior to my IQ project. 22. In the organization, policies and procedures were clear and well defined prior to my IQ project. 23. The organizational culture of the organization helped my IQ project succeed. 24. The organizational culture of the organization made it harder for my IQ project to succeed. 25. What is the primary cultural issue that helped my IQ project succeed? 26. What is the primary cultural issue that made it harder for my IQ project to succeed? 27. Choose the three most important issues to you and/or your project. 28. Choose the three least important issues to you and/or your project. 29. Are there other social issues or comments you would like to add regarding the impact of social issues on your project? 30. In what year was this project completed? 31. In which industry does your project organization belong? 32. Where was your organization located? Figure 1 – Survey Questions To ensure that the results from this study are statistically significantly different from random responses, an R script was used to generate a completely random sample (available in Appendix B). This involved generating numbers in the appropriate range for each question for 47 respondents. Once generated, a correlation matrix was calculated for all questions. This matrix, shown in figure 2, is clear that there is no correlation between any of the questions with the randomly-generated answers. Additionally, several statistical calculations were compiled for comparing the two samples. These can be seen in figure 3. Figure 2 – Correlation matrix for generated sample n mean sd median se Actual Generated Actual Generated Actual Generated Actual Generated Actual Generated X1 46 46 6.63 3.22 14.54 1.25 2.5 3.0 2.14 0.18 X2 46 46 4.20 2.78 0.96 1.33 4.0 3.0 0.14 0.20 X3 46 46 2.41 3.15 1.22 1.52 2.0 3.5 0.18 0.22 X4 44 46 3.20 3.35 1.13 1.40 3.0 4.0 0.17 0.21 X5 45 46 3.53 2.98 1.22 1.41 4.0 3.0 0.18 0.21 X6 45 46 3.84 3.26 1.09 1.48 4.0 4.0 0.16 0.22 X7 43 46 3.23 3.26 1.17 1.47 3.0 3.0 0.18 0.22 X8 44 46 3.14 3.22 1.03 1.41 3.0 3.0 0.15 0.21 X9 43 46 3.30 3.07 1.39 1.45 3.0 3.0 0.21 0.21 X10 45 46 2.42 3.09 1.45 1.47 2.0 3.0 0.22 0.22 X11 45 46 2.89 2.89 1.28 1.45 3.0 3.0 0.19 0.21 X12 43 46 3.30 3.07 1.32 1.54 4.0 3.0 0.20 0.23 X13 44 46 3.18 3.11 1.04 1.58 3.0 3.0 0.16 0.23 X14 44 46 3.80 2.80 0.85 1.44 4.0 3.0 0.13 0.21 X15 44 46 3.52 2.74 0.98 1.45 4.0 3.0 0.15 0.21 X16 43 46 3.70 2.87 1.01 1.59 4.0 3.0 0.15 0.23 X17 44 46 3.30 2.78 1.02 1.47 3.0 2.0 0.15 0.22 X18 45 46 3.20 3.04 1.01 1.44 3.0 3.0 0.15 0.21 X19 45 46 3.67 3.07 1.13 1.37 4.0 3.0 0.17 0.20 X20 45 46 3.62 2.72 1.05 1.38 4.0 2.0 0.16 0.20 X21 45 46 3.36 3.04 0.96 1.32 3.0 3.0 0.14 0.19 X22 44 46 3.27 2.93 0.95 1.37 3.0 3.0 0.14 0.20 X23 44 46 2.61 2.89 1.22 1.58 2.5 3.0 0.18 0.23 X24 46 46 1.61 7.30 0.93 4.02 1.0 7.5 0.14 0.59 X25 46 46 10.41 8.76 19.45 4.12 5.5 10.0 2.87 0.61 X26 46 46 12.41 12.87 14.67 6.65 13.0 13.0 2.16 0.98 X27 46 46 5.15 28.17 14.52 12.75 1.0 29.5 2.14 1.88 Figure 3 – Statistical Descriptions of the Actual and Generated Responses X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13 X14 X15 X16 X17 X18 X19 X20 X21 X22 X23 X24 X25 X26 X27 X1 1.00 0.06 -0.22 -0.22 -0.04 -0.26 -0.07 -0.10 -0.08 0.11 -0.18 0.26 -0.08 -0.10 0.04 -0.12 0.26 -0.04 -0.05 0.06 -0.13 -0.21 0.24 -0.09 -0.21 0.16 0.08 X2 0.06 1.00 0.09 0.08 0.31 0.14 -0.19 -0.27 0.00 -0.11 -0.15 -0.07 -0.07 0.01 -0.16 0.00 0.05 -0.13 0.02 0.11 -0.18 -0.04 0.06 0.06 0.12 0.00 -0.11 X3 -0.22 0.09 1.00 0.11 0.07 -0.19 0.15 0.08 0.02 -0.26 0.15 -0.23 0.00 0.38 -0.20 0.15 -0.12 0.34 -0.06 0.05 -0.10 0.31 -0.17 -0.05 -0.21 -0.19 0.12 X4 -0.22 0.08 0.11 1.00 -0.03 -0.07 0.08 -0.10 0.14 0.03 0.34 -0.17 -0.05 0.03 0.09 0.16 0.22 0.10 -0.15 -0.17 -0.18 0.34 -0.13 -0.11 0.07 -0.05 0.00 X5 -0.04 0.31 0.07 -0.03 1.00 0.01 0.10 -0.06 -0.06 -0.03 -0.11 0.01 -0.07 -0.01 -0.01 -0.08 0.09 -0.07 -0.09 0.03 -0.08 0.00 -0.01 0.10 0.00 0.22 -0.10 X6 -0.26 0.14 -0.19 -0.07 0.01 1.00 -0.12 -0.37 0.14 0.07 -0.08 -0.12 0.02 0.25 0.14 -0.09 0.01 0.17 -0.03 -0.01 0.04 0.05 0.12 0.34 0.07 -0.01 0.04 X7 -0.07 -0.19 0.15 0.08 0.10 -0.12 1.00 -0.03 0.19 0.14 0.18 -0.18 0.00 0.00 -0.04 0.19 -0.13 0.34 0.10 -0.01 0.03 0.12 -0.20 0.14 -0.16 0.03 -0.06 X8 -0.10 -0.27 0.08 -0.10 -0.06 -0.37 -0.03 1.00 -0.09 -0.18 0.09 0.02 -0.08 -0.04 0.31 0.00 -0.20 -0.09 -0.01 0.28 0.08 0.00 -0.03 -0.11 0.07 0.04 0.13 X9 -0.08 0.00 0.02 0.14 -0.06 0.14 0.19 -0.09 1.00 0.18 0.05 0.08 0.08 0.19 0.02 -0.06 -0.01 0.03 -0.07 -0.17 0.06 0.18 0.23 0.08 -0.03 -0.25 0.15 X10 0.11 -0.11 -0.26 0.03 -0.03 0.07 0.14 -0.18 0.18 1.00 -0.12 0.15 0.18 -0.14 0.00 -0.11 0.18 0.27 -0.05 -0.12 0.18 -0.01 0.21 0.38 -0.27 0.05 -0.18 X11 -0.18 -0.15 0.15 0.34 -0.11 -0.08 0.18 0.09 0.05 -0.12 1.00 -0.32 0.21 0.11 0.11 0.12 0.14 0.06 -0.01 0.15 0.03 0.23 -0.22 0.13 -0.14 0.03 0.21 X12 0.26 -0.07 -0.23 -0.17 0.01 -0.12 -0.18 0.02 0.08 0.15 -0.32 1.00 0.00 -0.32 -0.03 -0.20 -0.03 -0.03 0.04 -0.02 0.08 -0.29 0.15 -0.07 -0.13 -0.15 0.03 X13 -0.08 -0.07 0.00 -0.05 -0.07 0.02 0.00 -0.08 0.08 0.18 0.21 0.00 1.00 0.12 -0.14 -0.18 0.05 0.27 0.21 -0.03 0.32 0.08 -0.10 0.27 -0.10 0.38 0.15 X14 -0.10 0.01 0.38 0.03 -0.01 0.25 0.00 -0.04 0.19 -0.14 0.11 -0.32 0.12 1.00 -0.09 0.12 0.07 0.47 -0.14 0.09 -0.04 0.32 -0.15 0.19 0.01 -0.18 0.05 X15 0.04 -0.16 -0.20 0.09 -0.01 0.14 -0.04 0.31 0.02 0.00 0.11 -0.03 -0.14 -0.09 1.00 0.20 0.05 -0.04 -0.15 0.22 -0.01 0.05 0.01 0.15 0.12 0.13 0.00 X16 -0.12 0.00 0.15 0.16 -0.08 -0.09 0.19 0.00 -0.06 -0.11 0.12 -0.20 -0.18 0.12 0.20 1.00 0.11 0.10 -0.27 0.02 -0.04 -0.04 -0.16 0.11 -0.03 0.04 -0.08 X17 0.26 0.05 -0.12 0.22 0.09 0.01 -0.13 -0.20 -0.01 0.18 0.14 -0.03 0.05 0.07 0.05 0.11 1.00 0.05 -0.26 -0.06 0.09 -0.03 0.17 0.18 0.14 0.18 -0.07 X18 -0.04 -0.13 0.34 0.10 -0.07 0.17 0.34 -0.09 0.03 0.27 0.06 -0.03 0.27 0.47 -0.04 0.10 0.05 1.00 -0.05 0.07 0.06 0.38 -0.27 0.44 -0.18 -0.06 0.11 X19 -0.05 0.02 -0.06 -0.15 -0.09 -0.03 0.10 -0.01 -0.07 -0.05 -0.01 0.04 0.21 -0.14 -0.15 -0.27 -0.26 -0.05 1.00 0.15 -0.03 -0.20 -0.21 -0.21 0.03 -0.03 0.17 X20 0.06 0.11 0.05 -0.17 0.03 -0.01 -0.01 0.28 -0.17 -0.12 0.15 -0.02 -0.03 0.09 0.22 0.02 -0.06 0.07 0.15 1.00 0.02 -0.22 0.02 0.13 0.08 -0.10 0.44 X21 -0.13 -0.18 -0.10 -0.18 -0.08 0.04 0.03 0.08 0.06 0.18 0.03 0.08 0.32 -0.04 -0.01 -0.04 0.09 0.06 -0.03 0.02 1.00 -0.11 -0.14 0.35 0.16 0.16 0.02 X22 -0.21 -0.04 0.31 0.34 0.00 0.05 0.12 0.00 0.18 -0.01 0.23 -0.29 0.08 0.32 0.05 -0.04 -0.03 0.38 -0.20 -0.22 -0.11 1.00 -0.15 0.04 0.06 -0.21 0.13 X23 0.24 0.06 -0.17 -0.13 -0.01 0.12 -0.20 -0.03 0.23 0.21 -0.22 0.15 -0.10 -0.15 0.01 -0.16 0.17 -0.27 -0.21 0.02 -0.14 -0.15 1.00 -0.17 0.08 -0.09 0.19 X24 -0.09 0.06 -0.05 -0.11 0.10 0.34 0.14 -0.11 0.08 0.38 0.13 -0.07 0.27 0.19 0.15 0.11 0.18 0.44 -0.21 0.13 0.35 0.04 -0.17 1.00 0.09 0.21 -0.08 X25 -0.21 0.12 -0.21 0.07 0.00 0.07 -0.16 0.07 -0.03 -0.27 -0.14 -0.13 -0.10 0.01 0.12 -0.03 0.14 -0.18 0.03 0.08 0.16 0.06 0.08 0.09 1.00 0.03 0.03 X26 0.16 0.00 -0.19 -0.05 0.22 -0.01 0.03 0.04 -0.25 0.05 0.03 -0.15 0.38 -0.18 0.13 0.04 0.18 -0.06 -0.03 -0.10 0.16 -0.21 -0.09 0.21 0.03 1.00 0.01 X27 0.08 -0.11 0.12 0.00 -0.10 0.04 -0.06 0.13 0.15 -0.18 0.21 0.03 0.15 0.05 0.00 -0.08 -0.07 0.11 0.17 0.44 0.02 0.13 0.19 -0.08 0.03 0.01 1.00 In reviewing the calculations in figure 3, for all survey questions, each of the calculations vary between the actual responses and the generated responses. This shows a statistical difference between the actual responses and the randomly-generated answers. Results and Findings There were 46 complete responses to the survey representing projects in 11 states. The dates of the projects ranged from 2008 to 2014. This sample is not a true random sample and is not representative of the US population. It is, however, representative of the Masters in Information Quality Program at the University of Arkansas at Little Rock. In response to the statement “From an Information Quality standpoint, I was able to accomplish everything I desired with my project.” and, as seen in figure 4, only 4% disagreed. It must be remembered that this project was a requirement for the degree and this may have influenced responses to this statement. The next statements were utilized to determine if the respondents felt that, in general, cultural issues negatively or positively influenced the outcome of the project. As seen in figures 5 and 6, 10% felt there were negative influences and almost double of that believed there were positive cultural influences. As there were 14 Critical Cultural Success Factors (CCSFs) being investigated in this research, most of the survey questions dealt with these 14 CCSFs. Responses to these questions can be seen as follows in figure 7. Strongly Disagree 26% Disagree 35% Neither Disagree nor Agree 17% Agree 15% Strongly Agree 7% I believe that there were social/cultural issues NEGATIVELY impacting the success of my project. Strongly Disagree 9% Disagree 15% Neither Disagree nor Agree 31% Agree 30% Strongly Agree 11% No Response 4% I believe that there were social/cultural issues POSITIVELY impacting the success of my project. Strongly Disagree 4% Disagree 0% Neither Disagree nor Agree 11% Agree 41% Strongly Agree 44% From an Information Quality standpoint, I was able to accomplish everything I desired with my project.